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Abstract 

Background: Sequence symmetry analysis (SSA) is a potential tool for rapid detection of adverse drug events 
(ADRs) associated with newly marketed medicines utilizing computerized claims data. SSA is robust to patient 
specific confounders but it is sensitive to the underlying utilization trends in the medicines of interest. Methods to 
adjust for utilisation trends have been developed, however, there has been no systematic investigation to assess 
the performance of SSA when variable prescribing trends occur. The objective of this study was to evaluate the 
validity of SSA as a signal detection tool for newly marketed medicines. 

Methods: Randomly simulated prescription supplies for a population of 1 million were generated for two medicines, 
DrugA (medicine of interest) and DrugB (medicine indicative of an adverse event). Scenarios were created by varying 
medicine utilization trends for a newly marketed medicine (DrugA). In addition, the magnitude of association between 
DrugA and DrugB was varied. For each scenario 1000 simulations were generated. Average Adjusted Sequence Ratios 
(ASR), bootstrapped 95% confidence intervals (CIs), percentage of CI's which covered the expected ASR and percent 
relative bias were calculated. 

Results: When no association was simulated between DrugA and DrugB, over 95% of SSA CI's covered the expected 
ASR (ASR= 1) and relative bias was 1% or less irrespective of medicine utilization trends. In scenarios where DrugA and 
DrugB were associated (ASR = 2), unadjusted SR's were underestimated by between 1 1 .7 and 1 5.3%. After adjustment 
for trend, ASR estimates were close to expected with relative bias less than 1%. Power was over 80% in all scenarios 
except for one scenario in which medicine uptake was gradual and the effect of interest was weak (ASR = 1 .2). 

Conclusions: Adjustment for underlying medicine utilization patterns effectively overcomes potential under- 
ascertainment bias in SSA analyses. SSA may be effectively applied as a safety signal detection tool for newly marketed 
medicines where sufficiently large health claim data are available. 



Background 

Sequence Symmetry Analysis (SSA) has been suggested as 
a tool to complement current systems of post-marketing 
surveillance of medicines which use spontaneous report- 
ing databases [1]. The method, developed by Hallas [2], 
has been used increasingly with administrative claims data 
to investigate adverse effects of medicines including 
ace-inhibitor induced cough [3,4], inhaled corticosteroid 
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induced oral candidiasis [5], non-steroidal anti-inflamma- 
tory induced stroke [6], and isotretinoin and cardiovascular 
medicine induced depression [2,7]. A validation study, using 
known adverse drug reactions from randomized controlled 
trials as the gold standard, demonstrated that SSA has high 
sensitivity and moderate specificity for detecting safety 
signals [1], and had similar sensitivity and specificity 
to signal detection methods employed in spontaneous 
reporting databases. An advantage of the SSA method is 
its ease of application, computational speed and minimal 
dataset requirements. The method utilizes existing health 
claims datasets and due to the within person study design 
does not require numerical adjustment for time invariant 
patient specific confounders. 
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Sequence Symmetry Analysis assesses the association 
between two medicines in prescription claims data by 
comparing the sequence of initiation of each medicine 
during the study period or within a specified period of 
time for an individual. One of the medicines (DrugA) 
is the exposure medicine of interest and the other 
medicine (DrugB) indicates a possible adverse event 
for which a medicine may have been prescribed. In 
practice, SSA works by determining the first use of 
DrugA for an individual (ie the first supply date for 
DrugA in the available dataset for each individual). 
The same is done for DrugB. Then for each individual, 
DrugA and DrugB initiations within a defined period 
of time, for example 12 months, are selected and in- 
cluded in the analysis [8]. The ratio of the number of 
persons with DrugB initiated after DrugA is compared 
to the number of persons with DrugB initiated before 
DrugA. This ratio is described as the crude sequence 
ratio (CSR). If there is no association between the 
medicines the CSR will be approximately unity. If there is 
asymmetry in the distribution of initiation of DrugB 
after DrugA compared to before DrugA, it may imply an 
association between the medicines. The sequence ratio is 
an estimate the incidence rate ratio of the outcome event 
in exposed compared to non-exposed person time [2]. 
The within person study design ensures that the analysis 
is robust towards patient specific confounders that are 
stable over time, however, the analysis is sensitive to 
prescribing trends over time [2]. For example, if DrugA 
is decreasing in use but there is no trend in the 
utilization of DrugB over time it is likely that there will 
be more people starting DrugB after DrugA just by 
chance, even if the medicines are not associated. This 
may lead to an incorrect positive association between the 
medicines. Alternatively, an increase in DrugA over time 
will create an excess of patients with DrugA prescribed 
second and therefore an underestimate the association 
between DrugA and DrugB if one really exists. This latter 
scenario may happen, for example, when a new medicine 
enters the market. The implication of not allowing for 
prescribing trends would likely be an underestimate of 
a potential adverse event of a medicine. In order to adjust 
for such temporal trends, a null-effect sequence ratio is 
calculated. The null-effect sequence ratio is the sequence 
ratio that would be expected in the absence of a causal 
association, given the trend in incident medicine use in 
the background population [8]. The crude sequence ratio 
is adjusted for temporal trends in medicine use over time 
by dividing by the calculated null-effect sequence ratio. 

While the validity of SSA has been assessed using known 
adverse drug reactions as the gold standard [1], the validity 
of the method has not been assessed systematically using 
simulations studies where a known association is injected 
into the study population under varying scenarios of 



medicine utilisation. The advantage of simulation studies is 
that the validity of the sequence ratio can be determined in 
the presence of varying trends in incident medicine use and 
the validity of the null-effect sequence ratio in adjusting for 
temporal trends in prescribing can be assessed. 

The aim of this study was to evaluate the performance 
of SSA in scenarios which varied 1) the magnitude of 
the associations between the medicines using simulated 
prescription orders and 2) the trend in incident medicine 
use of a medicine new to market. 

Methods 

Sequence symmetry analysis was used to examine the as- 
sociation between DrugA and DrugB. The crude sequence 
ratio (CSR) was calculated as the ratio of the number of 
persons with DrugB after DrugA to the number of persons 
with the DrugB before DrugA. The null-effect sequence 
ratio (NSR) was calculated using the method described in 
Tsiropolous [9]. The adjusted sequence ratio (ASR) was 
calculated as the crude sequence ratio divided by the 
null-effect sequence ratio. We performed SSA on 1000 
simulations of each of the scenarios. Average ASR with 
bootstrapped 95% confidence intervals (CI) (500 replicates), 
the proportion of simulations in which 95% CI included 
the expected ASR (coverage probability), the proportion 
of simulations in which 95% CI did not include 1.00 
(power) and average relative bias (difference between the 
estimated sequence ratio and the expected sequence ratio) 
were calculated. 

Simulated data 

For a population of 1 million we randomly simulated 
prescription supplies independently for two medicines, 
DrugA and DrugB. For the population of 1 million, 20% 
of patients were randomly selected to initiate drugA and 
20% of patients to initiate DrugB during the study 
period. This meant that by chance 4% of patients would 
be initiated on both DrugA and DrugB. Since medicine 
supply dates were randomly generated, the number of 
patients prescribed DrugA then DrugB should be similar 
to the number of patients who were prescribed DrugB 
then DrugA. In this scenario the expected sequence ratio 
would be 1.0, that is, no asymmetry in prescription 
supplies. 

In order to generate the trend in medicine utilization, 
DrugA supply dates were randomly generated using the 
following distribution functions; uniform distribution 
for no trend, the log-normal distribution for a rapidly 
increasing trend, weibull distribution for a constant 
increasing trend and gamma distribution to simulate a 
gradual increasing trend in DrugA (Figure 1). In all sce- 
narios DrugB supply dates were randomly generated 
using the uniform distribution. 
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Figure 1 Simulated trends for incident use of DrugA and DrugB. 



We also simulated six scenarios in which the magni- 
tude of the associations between DrugA and DrugB were 
varied. In the first scenario, there was no association 
between the medicines and the expected sequence ratio 
was 1.00. In the other scenarios the expected sequence 
ratios were 0.6, 0.8, 1.20, 1.50, 2.00 and 3.00. The associ- 
ations were induced by forcing a percentage of patients 
who were not initiated on either medicine to be allo- 
cated a DrugA then a DrugB supply, in that order. SSA 
was restricted to sequences of medicine initiations 
within 12 months of each other as, when used in prac- 
tice, this is likely to limit the effect of age and other 
potential time-varying covariates on the probability of 
exposure and outcome. Sensitivity analyses were performed 
in which we reduced the prevalence of medicine use to 5%. 

SAS Version 9.4 was used for all simulations and 
analyses. 

Results 

Table 1 presents the simulation results of each medicine 
utilization scenario for the different effect sizes. 

Expected ASR = 1 .00 

When the expected sequence ratio was 1.0, corresponding 
to no asymmetry in the order of prescriptions supplied, 
Type 1 error was identified in less than 2.2% of simula- 
tions for all medicine utilization scenarios. When no trend 
was simulated the crude and adjusted sequence ratios 
were similar and relative bias was 0.38%. The crude 
sequence ratios were underestimated by 13.9%, 15.3% and 
11.7% when the simulated trend was gradual, constant 
and rapid respectively. After adjustment for trend using 



the null sequence ratio relative bias was reduced to 1% 
or less. 

Expected ASR = 1.2 

In the scenario where there was no trend in medicine 
utilization, 94.7% of CI's covered the expected ASR 
(ASR = 1.2) and power was 100%. When DrugA utilization 
was varied the CSR's were under-estimated by 11.9% to 
15.5%. After adjustment for trend, the mean ASR was 1.2, 
with over 94% of CIs containing the expected sequence 
ratio. In the case of a gradually increasing trend, power 
was only 66.4% compared to 96.0% and 96.8% when there 
was a constant or rapid increasing trend respectively. 
Relative bias was reduced to less than 1.5% in all scenarios 
after adjustment for trend. 

Expected ASR = 1.5, 2.0 and 3.0 

When there was no trend in medicine utilization, approxi- 
mately 95% of CI's covered the expected ASR and power 
was 100%. When DrugA was simulated to increase over 
time ASRs were close to the expected with relative bias 
1% or less. 

Expected ASR = 0.8 and 0.6 

When there was no trend in medicine utilization and the 
expected ASR was 0.6, 94% of CI's covered the expected 
ASR and power was 100%. When DrugA was simulated 
to increase over time ASRs were close to expected with 
relative bias 0.13 to 0.24%. Power was 86.3% when the 
expected ASR was 0.8 and the uptake in DrugA was 
gradual. Power was 99.9% for all other scenarios where 
the expected ASR was 0.8. 



Table 1 Sequence symmetry results of the 1000 simulations for different scenarios of medicine utilisation trends and effect sizes, for a population size of 1 
million and 20% population prevalence of use of each drug 
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Table 1 Sequence symmetry results of the 1000 simulations for different scenarios of medicine utilisation trends and effect sizes, for a population size of 1 
million and 20% population prevalence of use of each drug (Continued) 
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3 (0.73-0.88) 
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2054 


0.88 


0.70 (0.64-0.77) 


15.8 


100.0 


-12.17 


0.81 


3 (0.73-0.87) 


95.6 


99.9 


-0.07 



*Note: When the expected Sequence ratio =1, Power is the type I error. 



Table 2 Sequence symmetry results of the 1000 simulations for different scenarios of medicine utilisation trends and different effect sizes, for a population 
size of 1 million and 5% incidence of use of each drug in the population 
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0.84 
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78 
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0.53 (0.23-0.85) 


86.8 


77.9 


-9.71 


0.62 (0.27-1.00) 


95.2 


57.0 


5.92 


Constant 


136 


0.84 
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Table 2 Sequence symmetry results of the 1000 simulations for different scenarios of medicine utilisation trends and different effect sizes, for a population 
size of 1 million and 5% incidence of use of each drug in the population (Continued) 
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95.9 


16.6 


9.43 


Constant 


114 


0.84 


0.69 (0.43-0.97) 


79.5 


60.4 


-13.09 


0.82 (0.50- ' 


1. 15) 


95.0 


29.1 


3.14 


Rapid 


129 


0.88 


0.71 (0.46-0.98) 


83.2 


55.8 


-9.95 


0.81 (0.52-' 


1.12) 


94.1 


30.9 


2.47 



*Note: When the expected Sequence ratio =1, Power is the type I error. 



Pratt et al. BMC Medical Research Methodology 2014, 14:66 
http://www.biomedcentral.com/1471-2288/14/66 



Page 8 of 9 



Sensitivity analyses variation in medicine utilization 
prevalence 

The results of the analysis in which the population pro- 
portion of drug use was changed to 5% is summarised in 
Table 2. When the expected SR was 1.0, type 1 error was 
between 0.1 and 1.6% and relative bias was between 2.4 
and 12.2%. For an expected ASR of 1.2, power was 
20.8% when there was no trend in A, 8% when there was 
a constant or rapid trend and relative bias was between 
2.8% and 12.9%. For effect sizes 1.5, 2.0 and 3.0 when no 
trend was simulated, power was over 85%. For an effect 
size of 1.5, insufficient power was found for all trends. 
For an effect size of 2.0, power was less than 80% only 
for a gradual trend in DrugA. 

Discussion 

We tested SSA in different realistic scenarios for the 
underlying trend in uptake of a newly marketed medi- 
cine over time [9] for different effect sizes and a fixed 
population size of 1 million people. In all simulations 
adjustment for trends in prescribing using the null-effect 
sequence ratio appeared to effectively overcome under- 
ascertainment bias. When trends in medicine use were 
present, the crude sequence ratio underestimated the true 
association by 12-16%. After adjustment for underlying 
medicine utilisation patterns relative bias was 1 to 2%. 

SSA analyses had high statistical power in all simula- 
tions with effect sizes greater than or equal to 1.5 regard- 
less of the medicine uptake trend. For an effect size of 
1.2 and a gradual uptake of the medicine, SSA had only 
66% power. This is most likely a consequence of smaller 
numbers of patients available for analysis in the early years 
of a gradual medicine uptake rate. However, under the 
gradual medicine uptake utilization scenario, effect sizes 
of 1.5 and 2.0 both had nearly 100% power. Estimates 
of relative bias, however, were largely unaffected by 
sample size. These results suggest that SSA analyses may 
be a reliable method to identify adverse events associated 
with a newly marketed medicine in a sufficiently large 
population, particularly if the uptake of the medicine 
is rapid. 

In all simulations coverage probability was high as 
was the power. Coverage and power are measures that are 
dependent on the sample size. In a sensitivity analysis 
the impact of the prevalence of the medicine use on the 
performance of PSSA was explored by reducing the 
percentage of patients initiating the medicines to 5%. In 
general, the lower use of medicines reduced the power 
of SSA but increased the coverage probability margin- 
ally. This is most likely due to the increased variability 
and hence the width of the confidence intervals. The 
performance of SSA in terms of estimating the true 
estimated effect was slightly affected, as relative bias 
of the estimates increased marginally. These results 



suggest that the power calculations are dependent on 
the percentage of use and consequently the number of 
pairs generated. 

In practice, SSA has been shown to be robust to varying 
utilization patterns of medicine use [10]. An application of 
SSA to the investigation of the association between anti- 
psychotics and hyperglycaemia across six countries found 
a consistent positive association despite varying patterns 
of utilization in the different populations. The significance 
of the association was dependent on the number of pairs 
generated in each country [10]. In this study we have 
explored how varying trends in medicine utilization 
may impact on the validity of SSA, however, there may 
be other biases to consider when implementing SSA in 
practice including confounding by contra-indication and 
protopathic bias [2]. 

In this simulation study we fixed the population size at 
1 million and only considered a limited number of trends 
of medicine uptake in DrugA. In this analysis we only 
varied DrugA, however, in practice trends may occur in 
either or both DrugA and DrugB. Additionally, we only 
simulated scenarios where no association between DrugA 
and DrugB were present and where associations exist 
from 20% increased risk up to a tripling of risk and nega- 
tive associations where the expected ASR was 0.6 or 0.8. 
Future work will be required to determine the validity of 
SSA under conditions different to those explored here, 
such as varying population sizes, rates of initiation of both 
DrugA and DrugB and for more extreme associations. In 
particular, further studies will be required to determine 
the performance of SSA in situations where very rare but 
serious adverse events may be expected. In this simulation 
we have employed the method of adjustment as described 
by Tsiropolous [8]. This method is an amendment from 
the original technique first described by Hallas [2]. The 
rationale for this amendment was described in the paper 
by Tsiropolous to account for limited time intervals 
allowed between the exposure medicine of interest and 
the adverse drug event. This is relevant to the situation 
simulated in this analysis in which acute adverse events 
are of interest. In general the method of adjustment is 
dependent on the situation at hand and it may be more 
reasonable to use the method as originally described by 
Hallas. In a sensitivity analysis, the method as described 
by Hallas [2] was implemented and similar results were 
found (data not shown). 

A limitation of the SSA approach in practice is that it 
can only be applied to post-market surveillance issues 
where medicines are prescribed to treat the adverse event 
or where the outcome of interest may be identified as an 
admission to hospital. Examples of studies which have 
investigated medicine initiation as a proxy for an adverse 
medicine event include, insulin initiation as a proxy for 
acute hyperglycaemia associated with olanzapine initiation 
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[10], and antitussive medicine initiation as a proxy for 
cough associated with ACE inhibitors [3,4]. Results of 
SSA, like all observational study designs, must be inter- 
preted with appropriate consideration given to the sen- 
sitivity and specificity of the proxy medicine or adverse 
event hospital diagnosis as a measure of the adverse 
event of interest. 

Conclusions 

The results of this simulation study suggest that in practice, 
SSA may be a reliable tool to detect frequent adverse drug 
reactions when medicine uptake trends are variable. Use 
of SSA as a signal detection tool will help to ensure that 
adverse events associated with medicines are identified 
early and excess harm avoided. In the scenarios considered 
in this study, adjustment for underlying medicine utilization 
trends in a newly market medicine effectively over- 
comes under-ascertainment bias. The method identified 
an increased risk of one medicine, indicative of an adverse 
event, being prescribed after another in scenarios where a 
true association existed. These results suggest that SSA 
may be effectively applied as a safety signal detection tool 
for newly marketed medicines where sufficiently large 
health claim data are available. 
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